Skip to content

Fix: Decode single message over multiple stream reads#52

Closed
Phantom168 wants to merge 3 commits into
anthropics:mainfrom
Phantom168:fix/partial-json-in-buffer
Closed

Fix: Decode single message over multiple stream reads#52
Phantom168 wants to merge 3 commits into
anthropics:mainfrom
Phantom168:fix/partial-json-in-buffer

Conversation

@Phantom168

@Phantom168 Phantom168 commented Jun 30, 2025

Copy link
Copy Markdown

This PR aims to fix the issue claude code exiting due to parsing failure of a single json message over multiple stream reads Issue 6 Issue 15
The default maximum size of asyncio stream is 64KB. This limit can easily be breached when claude code invokes a tool to read a file of over 1000 lines.
Since a single message has been split over multiple stream reads, attempting to decode it as json leads to JSONDecodeError.

To demonstrate this, let's assume the asyncio stream limit is 5 characters, then the json message {"a": "bc"} would be split into 3 consecutive stream reads of {"a": , "bc" , }
Attempting to parse each of these individually will lead to JSONDecodeError error.

This PR includes a test case in test_subprocess_buffering.py to demonstrate the issue.
The test case fails without the fix.

To fix the issue, a buffer (accumulator variable) is initialised as an empty string. If there is a JSONDecodeError, the stream read is appended to the buffer. This cumulative buffer will be parsed. This will continue until the buffer has the complete JSON and is parsed successfully.

To avoid the buffer increasing infinitely, an arbitrary limit of 1MB (for now) has been put on the buffer, post which it'll result in an OverflowError.

@ltawfik

ltawfik commented Jul 1, 2025

Copy link
Copy Markdown
Collaborator

Thanks. Closing in favor of #53 which properly handles the buffer accumulation without the error handling and edge case issues found here.

@ltawfik ltawfik closed this Jul 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants